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Abstract. - In a community-structured population, public goods games (PGG) occur both within 
and between communities. Such type of PGG is referred as multilevel public goods games 
(MPGG). We propose a minimalist evolutionary model of the MPGG and analytically study 
the evolution of cooperation. We demonstrate that in the case of sufficiently large community size 
and community number, if the imitation strength within community is weak, i.e., an individual 
imitates another one in the same community almost randomly, cooperation as well as punishment 
are more abundant than defection in the long run; if the imitation strength between communities 
is strong, i.e., the more successful strategy in two individuals from distinct communities is always 
imitated, cooperation and punishment are also more abundant. However, when both of the two 
imitation intensities are strong, defection becomes the most abundant strategy in the population. 
Our model provides insight into the investigation of the large-scale cooperation in public social 
dilemma among contemporary communities. 
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, Introduction. — How cooperation emerges and pre- 
vails in a selfish population poses a challenging problem in 
evolutionary biology as well as behavioral science [Tlinj • A 
powerful paradigm for investigating this problem in groups 
of interacting players of arbitrary size is public goods game 
(■PGG) 12111]. In a PGG, each cooperator invests into a 
common pool while each defector attempts to exploit the 
public goods without any contributions. Thus, the payoff 
of a cooperator is always less than that of a defector. It is 
better off defecting than cooperating. 

During the past few years, a number of mechanisms 
have been demonstrated analytically or experimentally to 
promote cooperation [5^1 Ij . As an important mechanism, 
how spatial structure affects the evolution of cooperation 
has attracted much attention recently [T2Ul4j . In struc- 
tured populations, cooperators may form clusters to re- 
sist exploitation by defectors, resulting in the maintenance 
of cooperation. So far, most previous works of PGG in 
structured populations are based on lattice, small-world 
networks and scale-free networks. However, the study of 
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PGG in populations with community structure, which is 
a signature of the hierarchical nature of real social and 
biological systems |15l[TB] . has received little attention. 

The so-called community structure consists of many 
groups, where interaction rate within group is higher than 
that between groups |15II16| . Due to the community struc- 
ture, it is straightforward to consider that games are not 
only played among community members, but also played 
among different communities. Each individual engages in 
not only the "local" PGG in its community, but also the 
"global" PGG played among distinct communities. Hence, 
individuals are simultaneously involved in multiple PGGs 
on different hierarchical levels [T7Ul9j . These simulta- 
neous local and global PGGs in a community-structured 
population constitute a multilevel PGG (MPGG). 

Based on the MPGG, several straightforward questions 
arise: How to maintain cooperation in a large-scale among 
multiple communities? What is the effect of community 
structures on the evolution of cooperation? Some re- 
cent works have investigated the large-scale cooperation 
in PGG among contemporary societies by behavioral ex- 
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Fig. 1: (Color online) Sketch map of MPGG. On the first 
level, in each Gi -community, five individuals play a PGG to- 
gether; On the second level, in each G2-community, three Gi- 
communities are involved in a larger PGG; On the third level, 
two G2-communities take part in the largest PGG. 

periments [T7HT9] and simulation [20]. Some theoretical 
models are proposed to study the cooperation and pun- 
ishment in infinite group-structured populations by deter- 
ministic analysis |21| . However, finite population size is 
proved to bring internal noise which drives the population 
dynamics off the deterministic trajectory in the infinite 
situations Thus, a mathematical model of how 

community structures affect the evolution of cooperation 
in finite populations is still lacking. 

Motivated by these, we propose a minimalist evolution- 
ary model of MPGG and analytically study the evolution 
of cooperation in finite populations with such commu- 
nity structures where the interaction within community 
is far more frequent than that between communities. We 
adopt imitation updating rule and explore how the imi- 
tation strength within community and that between com- 
munities influence the evolutionary of cooperation. We 
demonstrate that under the condition of sufficiently large 
community size and community number, if the imitation 
strength within community is weak, or that between com- 
munities is strong, cooperation can prevail in the popula- 
tion. Nevertheless, if both of the imitation strengths arc 
strong, defection is the unique favorable strategy. Further- 
more, when the imitation within community is moderate, 
small imitation between communities may favor punishers 
prevailing while cooperators nearly disappear. 

Model. — Consider a finite population with commu- 
nity structures in which individuals take part in an n- 
level PGG. In this population, every nii individuals form 
a community, and any two such communities have no com- 
mon member. Denote this type of communities by Gi. 
Moreover, every 7712 Gi-communities constitute a larger 
community denoted by G2- This similar formation pro- 
cess repeats until to„ G„_i-communitics make up a G„- 
community which is the entire population. According to 
the above formation rule, this population is characterized 
by a hierarchical structure (see fig. [T]). 

We first study the simplest case with only two strate- 
gies: cooperation and defection (the case with punishment 



will be added and described later). A MPGG is played as 
follows: on the first level, in each Gi-community, nii indi- 
viduals play a PGG together. Each cooperator contributes 
c into the public pool in the Gi-community to which this 
player belongs, and every defector donates nothing. The 
total amount in this public pool is separated into two 
parts: one portion, whose proportion is fci, is allocated 
to the local PGG in this Gi-community, and the other 
portion, whose proportion is 1 — fci, is contributed into a 
higher public pool in the larger G2-community which con- 
tains this Gi-community. The total contribution in this 
local PGG is multiplied by an enhancement factor ri, and 
the product is distributed equally among all players in this 
Gi-community no matter whether they contribute or not. 

On the second level, in each G2-community, m2 Gi- 
communitics engage in a larger PGG. Each Gi-community 
contributes a fraction of the total amount (the proportion 
is 1 — fci), which is collected in the PGG among its mem- 
bers, into the public pool in G2-community which contains 
this Gi-community. Then, the total amount in this public 
pool in G2-community is also divided into two parts; one 
part whose proportion is ^2 is contributed to the PGG in 
this G2-community and the other part is submitted to the 
higher public pool in the Gs-community on the third level. 
The first part is multiplied by an enhancement factor r2, 
and the product is distributed among all individuals in 
this G2-community. 

Such type of PGG repeats until the highest level. On 
the highest level, the total amount in the public pool 
in G„-community is contributed into the global PGG. 
This amount is multiplied by an enhancement factor r„, 
then the product is distributed among the entire popu- 
lation. Although cooperators only contribute in the Gi- 
community on the lowest level, their contributions are al- 
located in n different PGGs at hierarchical levels. The 
payoff of each individual, irrespective of cooperators and 
defectors, is derived from n PGGs. 

Individuals in the population adjust their strategies 
through imitation. At each time step, two players i and 
j are randomly chosen. These two players belong to the 
same Gi-community with the interaction rate qi. The 
probability that individual i adopts the strategy of j is 
given by l/{l-|-exp[— wi(Fj — F,)]}, where Wi > denotes 
the imitation strength between two players in the same Gi- 
community, f j and Fj are the payoff of individual i and 
j [7j . The imitation strength measures the dependence of 
decision making on the payoff comparison. For wi 0, 
individual i imitates the strategy of j almost randomly, 
which is referred as "weak imitation". For — > 00, a 
more successful player is always imitated, which is referred 
as "strong imitation" . 

Moreover, if the two players do not belong to the 
same Gi-community, but they are part of the same G2- 
community, the interaction rate for these two players is 
92 ■ In this case, player i imitates the strategy of j with 
the probability l/{l + exp[—W2{Fj—Fi)]}, where W2 is the 
imitation strength between two players from different Gi- 
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Fig. 2: (Color online) Fixation process of a single mutant in 
a population, (a) A single mutant is produced in the popula- 
tion; (b) This mutant successfully takes over its Gi-community, 
and another individual from other Gi-communities imitates 
the mutant's strategy; (c) The mutant successfully invades its 
G2-community, and another one from other G2-communities 
imitates the strategy of the mutant. This type of fixation and 
imitation repeat until the population full of one type of players. 

communities but in the same G'2-community. In general, 
the interaction rate for two players belonging to different 
Gi-communities {I = 1, • • • , n — 1) but in the same G;+i- 
community is qi+i- The relationship J2?=i1i — 1 needs 
to be satisfied. In this case, player i changes its strategy 
to j's with the probability 1/{1 + cxp[— tiij+i (i^j — Fi)]} 
(I = — 1), where wi^i denotes the imitation 

strength between two Gj-communities. Since we focus on 
such community structure where the interaction within 
community is far more frequent than that between com- 
munities, we assume gi ^ 92 3> • • • ^ (/n. 

For finite populations, we analyze the advantage of a 
strategy through fixation probability, which measures the 
probability for a single mutant using such focal strategy 
to successfully take over the resident population. In or- 
der to obtain a general expression, we suppose there are 
two types of strategics, A and B. Imagine that a mu- 
tant adopting strategy A is produced in a population of 
B players. Since qi ^ 92, the time that this mutant 
takes over the Gi-community to which it belongs or dis- 
appears is shorter than that two individuals from different 
Gi communities meet. The time scales of fixation in a Gi- 
community and imitation between two individuals from 
different Gi-communities are separated. Thus, fixation 
of this mutant A in the population goes through several 
stages described by fig. [21 

Actually, the fixation process of a single A mutant in 
a population is equivalent to only n steps: the fixation 
of this A mutant in its Gi-community; The fixation of 
this Gi-community composed entirely of A players in its 
G2-community;...; The fixation of the G„_i-community 
invaded by the A mutant in the whole population. 

Denote the fixation probability of a single A mutant in- 
vading a Gi-community of B players by Pgj^. This fixation 
probability yo]^^ equals 



Pba= 
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i+E;=r' exp{^i ELii^M^i - ^) - mm" 
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where F\{i) and Fg(mi —i) are the payoff of each A player 
and each B player in the focal Gi -community, respectively. 



when there are i A players and nii — i B players in this 
Gi -community 

Denote the fixation probability of a Gi-community full 
of A players in its G2-community of only B individuals 
by p^^. Suppose there arc i Gi-communities consisting 
of only A players and m2 — i Gi -communities of only B 
players. In this focal G2-community, the payoff of each 
A player is denoted by F'^{i) and that of each B player 
is F^{m2 — «). A new Gi-community full of A players 
arises when two players with different strategies from dif- 
ferent Gi-communities are chosen, and the B player alters 
its strategy through imitation, then it takes over its Gi- 
community. Thus, the probability to increase the number 
of Gi -communities full of A players by one is given by 

Y+(i] = a Pba 

' ^^7^2 m2 l + cxp{~-W2[Fl{i)^ F^{m2-i)]}' 

Similarly, the probability to decrease the number of Gi- 
communitics full of A players by one is 



r^(i) = 92 



I 7712 — 1 



Pab 



7712 "12 1 + exp{~W2[F^{m2 - i) - F'^{i)]}' 



The fixation probability of a Gi-community full of A 
players in a G2-community is obtained as follows 



Pba-- 



1 



l+ET:l^Mw2Eum^2~^)-Fl{^mp\s/puy 



In general, denote the fixation probability of a single A 
mutant in a G/-community consisting of only B players 
(/ = 2, • ■ • , 71) by Accordingly, we have 



^BA ~ Pba ^ Pba x • • • x p^^. 



(2) 



where 



Pba= 



l+ET=l^MmEUnim-^)-F^AmiPAB/^^^^^ 



Two-level PGG with punishment. Wc now con- 
sider two-level PGG. Suppose three available strategies in 
this PGG: cooperation, defection and punishment. Pun- 
ishers are such type of players which contribute as coop- 
erators but reduce the payoff of defectors with a cost to 
themselves. We focus on the situation without second- 
order punishment which does not punish cooperators [7]. 

In each Gi-community, punishment acts as a personal 
behavior. Its object is defectors. Each punisher imposes a 
fine /3i on each defector at a cost 71 (71 < /?i ) to itself. The 
total fine for a defector relies on the number of punishers in 
this Gi-community, whereas the total cost for a punisher 
is determined by the number of defectors. 

Furthermore, for the separation of time scales, commu- 
nities always stay in homogeneous states. If a homoge- 
neous community is composed of punishers, they act as 
an institute of punishment. This institute of punishment 
punishes those communities consisting of defectors even 
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also containing cooperators or punishers since such com- 
munities free-ride on the global public goods. Specifically, 
a community full of punishers punishes those communities 
where defectors exist. Each punishing community reduces 
the total payoff of each punished community by mi/32, at 
a total cost TO172 (72 < /32)- Then, the cost of punishing 
is shared by all punishers in this punishing community, 
whereas the fine on the punished community is distributed 
among its members. Hence, the total fine for each individ- 
ual in the punished communities depends on the number 
of the punishing communities, while the total cost for each 
punisher in the punishing communities is determined by 
the number of the punished communities. 

Although the strategy updating is mainly dependent on 
imitation, mutation of strategies may happen sometimes. 
At each time step, every individual may mistakenly switch 
its strategy to a different and random strategy with the 
probability ^. Suppose the mutation rate /i — > 0. Suf- 
ficiently small /i assures that a single mutant vanishes or 
fixes in a population before the next mutant appears. The 
population is homogeneous most of the time jSl lMll^ . 
Therefore, in the limit of rare mutations, the evolutionary 
process of consideration can be approximated by a Markov 
chain where the state space is composed of homogeneous 
states full of each type of players. In this case, the state 
space of this Markov chain contains homogeneous state of 
cooperators, that of defectors and that of punishers. The 
corresponding transition probability matrix is 



The fixation probabilities $|)(^, ^cd^ *^i)P ^^"^ 
are given as follows: 



^^)p—PDP ^ Pdp 
1 



A= 



1 — <3)" —4)" 

^DC 
^PC 



^CD 
^ ^DC ^DP 
^PD 



^CP 

^DP 
1 — <J>" — <J>" 



(3) 



The normalized left eigenvector corresponding to the 
eigenvalue 1 of the matrix A determines the stationary 
distribution, which describes in the long run, the percent- 
age of time spent by the population in each homogeneous 
state. The stationary distribution for the above transition 
matrix eq. ([3]) can be calculated as follows 
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nc'^lp - 
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Xd = 


nc^D - 
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^p^c - 
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(4) 



where Xc, Xp), and Xp denote the probability to find the 
population in the homogeneous state consisting entirely 
of cooperators, defectors, and punishers, respectively, the 
normalization factor A insures Xc + Xpi + Xp = 1. 

We only discuss the situation of a two-level PGG in 
detail. In the case of no defectors, since punishers do as 
the same as cooperators under the condition of no second- 
order punishment, these two types of players are of no 
difference. This situation can be viewed as "neutral case" , 
where the fixation probability of a neutral mutant equals 
the reciprocal of the population size [53], that is, = 
1/(77117712) and ^^pc = 1/(71117712). 
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where 9 = c[w2{l — fci^i) -I- 7771(7111 — 1)]. 

Note that when wi ^ and 102 is not weak, the relation- 
ship ^cD < mirn2 ^i>c always satisfied in the case 
of fciTi > 1. Besides, when W2 00 and it;i is limited, 
there is ^jjc ^cd i'^ ^he case of fciri > 1. It indicates 
that in these two situations, cooperation is more abun- 
dant than defection [37]. Except for these two conditions, 
defection is always more abundant than cooperation. In 
addition, when 771171 3> c and mi(3i 3> c, the inequal- 
ity ^Dp > ^pjj is always satisfied. Furthermore, when 
'T1272 3> c — cfciri and m2/32 3> cfciri — c, the relation- 
ship pjjp > Ppp) always holds, regardless of the imitation 
strengths wi and 7112. Hence, if mi and 7712 are sufficiently 
large, punishers are always more abundant than defectors. 

Based on the stationary distribution, we find that 
when 7711 and 7712 arc sufficiently large, weak imitation 
within Gi-community or strong imitation between Gi- 
communities is of great benefit to cooperators and pun- 
ishers (see fig. [3]). In these two cases, cooperators do as 
well as punishers, they are both more abundant than de- 
fectors. However, if these two imitation strengths are both 
strong, it is harmful to the evolution of cooperation and 
punishment. Moreover, in the case of moderate imitation 
strength wi, small W2 may favor punishers prevailing but 
has a little effect on the emergence of cooperation (see 
fig.|31Ja)). Furthermore, when W2 is moderate, the preser- 
vation of cooperators and punishers are hindered. When 
7«2 is large enough, cooperation and punishment arc still 
more abundant than defection. 

The reason for the above phenomenon is that under the 
condition fciri > 1, weak 7771 incurs ^%c ^cr" ^^'^ the 
inequality ^j^p > always holds for sufficiently large 
nil and TO2- Based on eq. we obtain Xc > Xp) and 
Xp > Xp) for weak wi. In this case, the population spends 
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Fig. 3: (Color online) Stationary distribution as functions of 
imitation strengths wi and W2- (a) Probability Xc; (b) Prob- 
ability Xp. Probability Xd can be derived from Xc + Xd + 
Xp = 1. Weak wi or strong W2 is favorable for the prevalence 
of cooperators as well as punishers. However, if both imitation 
strengths are strong, it is harmful to the emergence of cooper- 
ation and punishment. Parameters: c = 0.8, k\ = 0.5, ri = 3, 
/3i = 1, 71 = 0.5, /32 = 1, 72 = 0.5, mi = 20, and m,2 = 20. 

most time in homogeneous state of cooperators or punish- 
ers. We state that our below results are all based on the 
assumption of sufficiently large toi and 7712. Note that fciri 
denotes effective enhancement factor within community. 
Only when the effective enhancement factor larger than 
one, cooperation may be favored in the long run. This 
condition fciri > 1 is consistent with that in |2- More- 
over, when wi is moderate (wi = 0.1 in fig. IH^a)), small 
W2 leads to the inequality < ^^u- However, there is 
always ^\,p > It indicates that defectors are more 

abundant than cooperators while punishers arc superior 
to defectors. Note that punishment and cooperation are 
equal. Which one is the most favorable strategy depends 
on the comparison between ^cjj and $|)p. Denote the 
ratio ^cD/^i)P K. From fig. HJb), the rising K leads 
to decreasing Xp as well as Xc, but increasing Xo. When 
the gap between and $|)p is sufficiently small, the 

population spends its most time staying in the homoge- 
neous state of punishers. With the increase in this gap, 
the advantage of defection over cooperation is enhanced, 
or that of punishment over defection is weakened. Con- 
sequently, defectors become more and more frequent than 
punishers and cooperators. However, when the imitation 
strength W2 reaches so large that makes the inequality 
^iic ^ ^CD satisfied, defectors perform the worst, the 
population is most likely to be found in the homogenous 
state full of cooperators or punishers with nearly equal 
probabilities. 

Large imitation strength W2 can also be viewed as posi- 
tive out-group attitude which shows preference for individ- 
uals from other communities, while weak 'W2 can be seen 
as neutral out-group attitude. From fig.lUJa), neutral out- 
group attitude is of great benefit to punishment but harm- 
ful to the evolution of cooperation. The impact of positive 
out-group attitude on the evolution of punishment is com- 
plicated. With an enhanced positive out-group attitude. 
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Fig. 4: (Color online) (a) Stationary distribution as a function 
of imitation strength W2 in the case of moderate wi {— 0.1). 
Strong W2 makes cooperators and punishers more abundant 
than defectors. Besides, small W2 also promotes punishment 
to be the most abundant strategy. However, moderate W2 is 
harmful to both cooperators and punishers. (b) The ratio be- 
tween s-i^d as a function of imitation strength W2 
when wi =0.1 in the case of < ^cd- The ratio K sus- 
tainedly rises. It leads to the decrease of Xp as well as Xc 
and the increase of Xd when < ^cd- Parameters in 
these two figures are the same as those in fig. [3l 

punishment is favored at first, then its amount shrinks and 
rises finally. When the preference of individuals for those 
in other communities is sufficiently large, cooperation is 
also greatly favored. 

Discussions and Conclusions. we have proposed 
a minimalist theoretical model of MPGG in finite pop- 
ulations with community structures and explored under 
what circumstances the assortment of cooperation can be 
achieved in community-structured populations. We found 
that if the community size and the community number 
are both sufficiently large, weak imitation within commu- 
nity or strong imitation between communities promotes 
the prevalence of cooperation. This can be attributed to 
the principle that weak imitation within community may 
lead to assortment of cooperators, while strong imitation 
between communities assures the prevalence of coopera- 
tive behavior once a cluster of cooperators appears. How- 
ever, if the imitation strengths within and between com- 
munities both become strong, cooperation as well as pun- 
ishment are eliminated from the population. In addition, 
it is interesting that when the imitation within community 
is moderate, small imitation between communities makes 
punishers extraordinarily abundant in the population but 
cooperators nearly disappear. 

A model relevant to ours is from ref . |28j , where Traulsen 
and Nowak studied the effect of multilevel selection on the 
evolution of cooperation in Prisoner's Dilemma (PD), a 
classic two-person game. Compared with this model, we 
focus on PGG, a multi-person game, which has different 
ingredients and background from PD. Moreover, in [55], 
the game only occurs in each group, and there is no inter- 
action between any two individuals from different groups. 
However, in our model, game exists not only in each com- 
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munity but among different communities. Besides, inter- 
actions always happen between individuals from distinct 
communities. It incurs the prevalence of a strategy across 
community. For the strategy updating rule, Moran pro- 
cess is applied in [28) while we adopted imitation process. 
Although the approximate expressions of fixation proba- 
bility in these two different processes in the limit of weak 
selection arc almost identical [53], those in the case of a 
moderate selection are extraordinarily different from each 
other. In addition, according to the evolution process, we 
obtain the essential difference between the model in [55] 
and ours as the mechanism to promote cooperation: the 
former is group selection whereas the latter is spatial se- 
lection. Group selection is suitable for the situation where 
individuals compete within groups and groups also com- 
pete with each other; Spatial selection is valid when there 
is only assortment of cooperators and no group level of se- 
lection |30| . In our model, there is no competition among 
communities and no selection at group level. Thus, this is 
not group selection but spatial selection. 

Another similar concept to the community-structured 
population in biology is metapopulation |31j . Although 
both metapopulation and the community-structured pop- 
ulation can be viewed as group-structured population, the 
mechanisms for the evolution of populations in these two 
types of models are different. The evolution of species 
in metapopulation is driven by recolonization and extinc- 
tion, i.e., birth and death process in biology, while the 
evolution of cooperation in our work is inspired by imi- 
tation which is a behavior in sociology. In addition, we 
consider a simple type of punishment in this paper, which 
solely punishes defectors. In this case, cooperators become 
second-order free-riders since they exploit the sacrifice of 
punishers. Thus, cooperation should also be punished. 
Sigmund et al. shew that incorporating second-order pun- 
ishment, which punishes both defectors and cooperators, 
the evolutionary dynamics can be drastically altered [3- 
Besides, amount of empirical evidences reveal that defec- 
tors sometimes punish cooperators [31]. The correspond- 
ing population dynamics can be qualitatively changed by 
this "anti-social punishment" [33]. Therefore, to explore 
the effects of second-order punishment and antisocial pun- 
ishment on the evolution of cooperation deserves more at- 
tention in future studies. 
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